A Convergent Incremental Gradient Method with a Constant Step Size
نویسندگان
چکیده
منابع مشابه
A Convergent Incremental Gradient Method with a Constant Step Size
Abstract. An incremental gradient method for minimizing a sum of continuously differentiable functions is presented. The method requires a single gradient evaluation per iteration and uses a constant step size. For the case that the gradient is bounded and Lipschitz continuous, we show that the method visits regions in which the gradient is small infinitely often. Under certain unimodality assu...
متن کاملConvergence diagnostics for stochastic gradient descent with constant step size
Iterative procedures in stochastic optimization are typically comprised of a transient phase and a stationary phase. During the transient phase the procedure converges towards a region of interest, and during the stationary phase the procedure oscillates in a convergence region, commonly around a single point. In this paper, we develop a statistical diagnostic test to detect such phase transiti...
متن کاملA stochastic gradient adaptive filter with gradient adaptive step size
This paper presents an adaptive step-size gradient adaptive filter. The step size of the adaptive filter is changed according to a gradient descent algorithm designed to reduce the squared estimation error during each iteration. An approximate analysis of the performance of the adaptive filter when its inputs are zero mean, white, and Gaussian and the set of optimal coefficients are time varyin...
متن کاملA globally convergent incremental Newton method
Motivated by machine learning problems over large data sets and distributed optimization over networks, we develop and analyze a new method called incremental Newton method for minimizing the sum of a large number of strongly convex functions. We show that our method is globally convergent for a variable stepsize rule. We further show that under a gradient growth condition, convergence rate is ...
متن کاملSolving the liner quadratic differential equations with constant coefficients using Taylor series with step size h
In this study we produced a new method for solving regular differential equations with step size h and Taylor series. This method analyzes a regular differential equation with initial values and step size h. this types of equations include quadratic and cubic homogenous equations with constant coeffcients and cubic and second-level equations.
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: SIAM Journal on Optimization
سال: 2007
ISSN: 1052-6234,1095-7189
DOI: 10.1137/040615961